Reliable Robustness Evaluation via Automatically Constructed Attack Ensembles

Attack Ensemble (AE), which combines multiple attacks together, provides a reliable way to evaluate adversarial robustness. In practice, AEs are often constructed and tuned by human experts, however tends be sub-optimal time-consuming. this work, we present AutoAE, conceptually simple approach for automatically constructing AEs. brief, AutoAE repeatedly adds the attack its iteration steps ensemble that maximizes improvement per additional consumed. We show theoretically yields provably within constant factor of optimal given defense. then use construct two l∞ l2 attacks, apply them without any tuning or adaptation 45 top defenses on RobustBench leaderboard. all except one cases achieve equal better (often latter) robustness evaluation than existing AEs, notably, in 29 best known one. Such performance shows itself as protocol robustness, further indicates huge potential automatic AE construction. Code is available at https://github.com/LeegerPENG/AutoAE.